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ABSTRACT 



individual 84)8s<Lon0 vjoso 6ondttct»cd wilxlx 

two 10th grade students in a large inner-city high school/ 
with the purpose of revising a unit of M «.4.Uv»iiaU (!S 

from a standard text, and testing the hypothesis that 
material so revised would yield more learning for students 
of this population them materials revised under standard 
curriculum workshop conditions. After each tutorial 
session, text material was revised on the basis of several 
student variables* reading difficulties, verbalized con- 
ceptual difficulties, answers to diagnostic questions, 
and responses to tutors' questions. After twenty-three 
such revisions, when it was demonstrated that typical 
students were able to learn the material, two groups of 
fifty students each, matched on IQ, arithmetic and reading 
if ' i achievement scores, were presentdd with the unit. The 

Experimental group studied tutorially revised materials, 
and the Control group studied teacher-prepared materials 
based upon the same unit. Students in the Experimental 
group scored significantly higher ( ; .025) on criterion 
tests administered immediately on completion of the material. 
The hypothesis that tutorial revision of instructional 
material will produce a more effective teaching tool than 
will workshop (teacher) revision is supported. 

From the recorded observations of student learning problems 
during tutorial sessions, an index was prepared of the types 
of problems encountered, the specific instructional materials 
causing the problem, and the strategies used to resolve it. 
This index was coded for later inclusion in a computerized 
data file to be used in the training of teachers by simu- 
lating actual educational interactions. 

Recommendations were made, on the basis of the experience 
gained in this feasibility study, for a major research 
project aimed at the production and validation of five 
semester units of instruction in subject areas basic to 
programs of vocational education. 
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INTRODUCTION 

The research program here reported was designed to test two 
methods of revising curriculum materials in order to tailor them 
to the learning needs of a special target population of students. 
The study was anticipatory of a larger research program, designated 
Phase II, which will have the purpose of comparing and recommending 
three procedures for the validation of textbooks for deprived, 
inner-city students and for preparing multi-media packages ©f —if 
instructional materials in five subject areas basic to success 
in vocational education courses. The hypothesis under test in the 

°tudy being reported contends that instructional material re- 
vised in iteratively cycled individual tutorial sessions with 
students from an urban deprived population would yield more learning 
than would material revised in a typical curriculum workshop situa- 
tion involving modifications based upon the teachers subjective 

s tuten^ aydte* aP ° n Air * et interac tion with the individual 
evaluation and into 0,6 feedback loop of material 

more powerful instructional SlSSeST"** 18 ' 4t * 88UMea that 
face to face interaction between the 

Uonal material. Additional data sought during the pilot phase 
were, measures on various aptitude and achievement tests, criterion 
test responses; and observer notes on program content and inter! 
actions between students and material, a procedure for observing 
coding and classifying this data was to be standardised during 
the study and recomnendations developed for coding procedures to be 
followed during the main research program or „ b 

From the experience of those teachers directly &* V oi a 
the tutorial sessions, a technique for the training of tJ^ * th 
for these deprived students has been recommended, in Phase x* 
computer-based simulation technique will be evolved whereby actua. 
teaching interactions will be simulated and teacher trainees will be 
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required to reset as Uwugh in the actual setting. This will pro- 
vide an economical and convenient method to expose teacher trainees 

to the particular learning difficulties associated with this s tuden t 
population. 



Concurrent to the pilot research , curriculum workshops have 
met to develop instructional units in five different areas. These 
units will be employed in the main research project and represent 

a product similar to that developed by text publishers or curricu- 
luro developmen^institutes . 

This pilot study and the stain research effort comprise an 

evaluation of different models which might be adopted 
by any local school district for tailoring curriculum materials 
to educate special student populations. The majority of conmercial 
texts are aimed at a wide representative sample of students and may 
be entirely inadequate to the learning needs of select populations? 
vis, the urban disadvantaged. Through the development of a 



standard methodology for evaluating, modifying and preparing in- 
dividualised study programs for inner city student, the educational 

needs of this population of disadvantaged children might someday 
be met. 



METHOD 

Design ? 

The design of this study consists of comparisons of learning 
test scores between groups of students studying tuisorially revised 
material and a group studying workshop (teacher only) prepared 
materials. The students receiving the workshop material compose the 
Control group. There were three experimental groups which studied 
the tutorially revised material! the Laboratory, Audio, and Class- 
room groups. The Laboratory group were administered their education- 
al material in a tutorial center in their high school. The Class- 
room group received the material in their own classroom. The 
Audio group was run in the tutorial center and received the same 

material but the method of its presentation was varied; it was 
recorded on 













audio tapes and they simultaneously listened to and read it. All 
materials were presented in a self -instructional format. It was 
hypothesised that the groups studying the tutorially revised mater- 
ials would have significantly higher learning scores than the group 
studying the workshop prepared materials, at an alpha level of ,05, 
The study was conducted in Simon Grats High School, Philadel- 
phia, Pennsylvania, All subjects were tenth graders currently 
enrolled in a course in general mathematics. No one from the 
“modified 11 program for low I.Q. students was included, as an arbi- 
trary cut off at a measured X«Q« of 80 was made . Fifty subjects 
each were included in the Laboratory and Control Groups, They 
were chosen at random from classes meeting at five different peri- 
ods a day and were then matched on I.Q. , and arithmetic and reading 
achievement test scores. One member of each pair was then randomly 
assigned to the experimental group; the other to the control group. 
Nineteen subjects were included in the Audio group; these students 
too were chosen at random from classes meeting at five successive 
periods a day. in the Classroom group, twenty-five students were 
run. All subjects in this group received the material at the same 
time, the group was chosen specifically because the teacher was 
willing to give up her class for the time needed to run the experi- 
ment. Attendance records became an additional criteria of selection 
No student who had missed more than 30 days in his previous year 
at school was considered for either group. This further narrowed 
the field of choice. 

Tutorial Revisions 

Tutorial sessions were conducted to revise the basic text 
materials. Individual sessions were initiated using a unit on 
"The Measures of Central Tendency" from the tenth grade General 
Mathematics, Book One , by Brown, Simon and Snader. Instruction 
was in the Simon Grats tutorial center and usually lasted a full 
period (forty-five minutes) per child. 








Tutorial subjects were volunteers, as representative as poss- 
ible ©£ the experimental and control groups. Tutorial sessions 
were held, for the most part, six periods each day. The first 
three periods were devoted to tutoring one of the revisions; the 
intervening time was given over to developing a new revision for 
the afternoon students; and the last three periods of the day to 
tutoring the resulting revision. Because each new revision required 
re-typing, and also grew in length, it soon became apparent that 
only three students a day could be comfortably handled with the 
available staff, in all forty-two students were cycled through 
the tutorial procedure. This required two months time and engendered 
twenty-three revisions of the original mat .rial. The final revision 
required approximately four days for a student to complete. 

In the first tutorial sessions the students were required 
to read the text material aloud and were encouraged to talk about 
any conceptual difficulties encountered. Because of the reluctance 
of the students to verbalize any learning difficulties, diagnostic 
questions were built into the text at the end of every instructional 
paragraph. Text exercises were used and student errors analyzed. 

The tutor spontaneously questioned the student whenever it was 
evident there was a difficulty and attempted various teaching tech- 
niques to communicate the material. After each group of tutees had 
been through their daily sessions, the researchers worked out the 
next revision on the basis of the behavior that had been observed 
and the strategies that had appeared to be particularly effective. 
Essentially, this procedure was maintained through to the twentieth 
revision. From the twentieth revision on, the material was admin- 
istered in a self- instructional, rather than a tutorial, format as 

this was the method to be employed in the actual experimental 
situation. 

Coding Procedures 

During the tutorial sessions, an observer took notes on all 
the difficulties the students encountered with the material. The 
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noteg were written on a copy of the particular revision involved 
O' 80 that the specific material engendering each problem could be 

identified easily. The notes were transcribed into a list of 
Student Problems ' 1 and the list was categorize ; d in terms of the 
kinds of problems involved. Also, a tabulation was made of every 
problem every student encountered during every session. During 
the tutorial sessions, tape recordings were made of the proceedings 
to be used where needed to validate the notes of the observer. 
Jpendix A gives examples of the observations of one student taking 
the fifteenth revision. In Appendix B, several general instructional 
concepts are listed and typical learning problems they generated 
are identified. 

notes on each student can be coded according to types 
of problems encountered and these data then used as input to a 
computer. Once put on magnetic tape, the student problems will 
be tallied and breakdowns made according to individual students, 
number of particular revision, and certain student variables (read- 
ing level, arithmetic achievement score, I.o. , etc.). Analyses of 
1 these data will provide information on typical learning difficulties 

of individual students and on the teaching efficacy of each serial 
revision of the instructional material. This data bank can also 
prove of value in the training of teachers by simulating actual 
educational interactions. 

Text Revisions 

Changes in the original text and in the subsequent revisions 
were made on the basis of several student variables. These were: 
reading difficulties the students encountered; conceptual difficul- 
ties verbalized by the students; answers to diagnostic questions, 
both correct and incorrect? and responses to tutor's questions* 
Revisions of the material were designed to facilitate student 
learning by anticipating the types of difficulties likely to be 
encountered and providing instruction to forestall them. The 
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eeision to 3 t°p revising the tent material was made when it appeared 
^at typical students in this popuiation were able to ieam aU the 

to a T h °T Xn0d in the Units When th ®y «»« respond correctly 

TJlT t 2T etio questions in the text witteut - 

Several general instructional variables were manipulated in 
the successive revisions in an attempt to write a revision that 
communicated effectively. These variables included general format 
of presentation of material, order of presentation of material; 
vocabulary level; amount of material presented in each frame; step 

- aaK>unt of conceptual progress between frames; speed of 

ding of each concept; and amount of repetition. These variables 
f considered in relation to the specific material being 
taught, see Appendix 3 for a comparison between some of the revi- 
sions, and Appendix D for the final revision. 

Wo rkshop Productlon _of_ Materlal for Instruc t ion of Control Group 

the mathematics teacher from Simon Grats prepared 

the unit to be taught to the control group. This material is based 

in «,! rr r! fc ' fr ° m tte Same textbook ' «« that Which was revised 
in the tutorial sessions. The teacher first taught the unit in one 

of her classes and then reworked the text material into a complete 

seif-instructional unit. At her discretion, text material was 

augmented, and improved. This technique represents the rational 

Z r ?!f .° rdina ! llY ^ by teXtb °° k autho ~ »»* teacher institutes 
devoted to writing instructional material. See Appendix E for the 

completed text prepared by the workshop committee. 

Criterion Testify 

^ * f r th<S tufcorlal visions and the workshop revisions were 
initiated, a criterion test was developed by the tutor and the 

workshop teacher. The test measured the behaviors to be taught in 
the unit. After all revisions were completed, the revised material 
was administered to groups of subjects who studied it and then took 

/ 
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the criterion test. The workshop produced material was used by the 
Control group. The three experimental groups studied the tutorially 
revised material. Both instructional units were administered to 
50 Laboratory and 50 Control subjects in the laboratory setting of 
the Simon Grata, tutorial center. Approximately twenty subjects 
were run at a time; all of them received the materials in a self- 
instructional format. The administration of the materials occupied 
approximately six school days or -until the last student had comple- 
ted the unit and the criterion test. Each student worked at his 

own pace, spending one class period (forty-five minutes) per day 
in the tutorial centers 

Two other experimental groups were also run using the tutori- 
ally revised material. One group, the Audio group, was run in an 
attempt to compensate for the verbal difficulties of this population 
of students; they simultaneously received the printed material and 
listened to an audio tape recording of the material. A total of 
nineteen students were in this group? they studied four at a time, 
each having his own tape recorder and ear phones. Stops were 

programmed onto the tapes at those points where the students were 
supposed to do actual work. 

The other experimental group received the tutorially revised 
material in the printed format in their classroom. The students 
did not study the material or take the test in the laboratory set- 
ting of the tutorial center. This group was designated the Class- 
room group and consisted of twenty- five subjects. 

Appropriate t tests were made between the differences in 
learning scores of all four groups? The Laboratory, the Control, 
the Audio, and the Classroom groups. 

Other Workshop Sessions 

During the course of this study, five other workshop sessions 
were conducted to prepare material in five curriculum areas. This 
material is to be used in the second phase of the study and includes 
units in Composition Writing, Speech, Reading, General Mathematics, 
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and Manipulative Skills* Each workshop group consisted of an exper~ 
ienced teacher in the area, a consultant, and assistants who were 
college seniors or graduate students in education specializing in 
the subject matter of that specific unit. Each of the five groups 
analyzed and prepared an instructional unit and wrote the oonoorni — 
tant exams, completing the steps listed below: 

1* an explicit statement of the test to be used and 
the material to be covered in that text, 

2* a definition of the behaviors taught in the unit, 

3* a pre-test to evaluate whether the student had 
the skills -necessary to take the unit, 

4* within-unit diagnostic quizzes to determine the 
students' comprehension of the material and there- 
fore their readiness to go on to the next step of 
instruction, 

5* a criterion test to determine the level of learning 
after the student had completed the unit, 

6* a retention test, 

7. actual revision of the text material thrr>”»’ 
additions of necessary material , yo**A«J.on of 
vocabulary that might bo boynnd the student's 
grasp, deletion of oxtraneous material, etc. 

Appendix C gives samples of this material. 

Participants in the workshop sessions received two weeks of 
instruction in programming , i.e. , defining instructional goals, 
preparing alternative instructional sequences, establishing test 
validation procedures, etc. Or. James Evans, Vice President of 
Teaching M aterials Corporation, and Mrs. S. K. Dunn of the Office 
of Economic Opportunity both served as alternate instructors. 

Each workshop team then proceeded independently on a twice weekly 
schedule to prepare their instructional units. However, frequent 
interaction among the different teams became the rule during the 
planning sessions. Consultants Temple and Pennsylvania Universities 
were als^ shared among the various teams. 



Several students of the Simon Gratz high school population 
were involved in a preliminary validation of the units on math 
and on manipulative skills. In general, however, teachers partic- 
ipating in the workshop refrained from using or testing 

their materials during the workshop developing* v r 

RESULTS 

Descriptive Data 

Certain aptitude and achievement test scores were available 
on the subjects used in this study. For all subjects an I.G* 
measured on the Philadelphia Verbal Intelligence scale was available 
all subjects also had a score on an Arithmetic Fundamentals test; 
and data were available for most subjects on a Reading Fundamentals 
test. The information on the last xs in the form of two 

scores, one for Reading mid one for Usage. These data are presen- 
ted in Table 1. Both the Arithmetic and Reading tests were admin- 
istered in the eighth "grade; by halving the scores a grade level 



score is obtained. Table 2 presents these grade levels. 

TABLE Is Aptitude and Achievement Test I leans by Group 

TEST 


GROUP 


f*hila. 

JLOi 


Arithmetic 

Funds. 


Reading 

Reading 


Funds. 

Usage 


Control Group (N«50) 


68.14 


12.94 


14.89 


14.16 


Laboratory-experiment- *x 

u roup 


8g- 26 


13.10 


14.17 


13.95 


Rudio-*r- -‘“ ental 

(N-19) 


88.42 


13.00 


14.32 


14.31 


«*8room-experimental 
Group (n«25) 


92. 54 


12.90 


13.89 


13.69 






TABLE 2: 
GROUP 

Control 

Laboratory 

Audio 

Classroom 



Achievement Test Grade Level Means by Group 

— T E S T 

Arithmetic Fundamentals 

6.47 
6.55 
6.50 

6.48 



Reading Fundamentals 
7.46 
7.05 
7.16 
6.95 



For the students to have been achieving at grade level, all 
e Arithmetic and Reading test score means should have been 16.00. 
The tabled means represent from 1/2 to 1-1/2 years lag in achieve- 



In 1964 the Philadelphia Research Division of the Board of Ed- 
ucation conducted an achievement testing program. Prom this 
the eighth grade Arithmetic Fundamentals mean grade level for the 
lowest performing school school in the district was 6.3. Ill low- 
est performing school mean grade level for the eighth grade Reading 
undamentals was 7.1. The students in the samples used in this 
study have achievement scores on the average that are on a par with 
those from the lowest achieving schools in the district. 

students are on the bottom when compared with their whole school 
district. 

The I.Q. scores are equally depressed, representing an average 
of 10 to 21 plus points below the average score of 100. According 

to their test scores, these students must be classified education- 
ally handicapped. 

The corresponding data for the students used in the tutorial 

sessions to develop the experimental revisions are presented in 
Table 3. 



TABLE 3 2 Aptitude and Achievement Test 
Means for Tutorial Subjects 



TEST 



Phil a. 
I.Q. 

Tutorial Subjects (N»42r"9'0.00 



Arithmetic 
Funds. 
*1X35 



Reading Funds. 
1 10 5 ' — 



Criterion Tests 



The three experimental groups and the Control group were ^iven 
the same criterion test in o*** r nieaeure learning posfociwuiCG • 
The means for these groups are prebonteci in Table 4. 



TABLE 4? Group Means on Criterion Test 



GROUP 


MEAN CRITERION TEST PERFORMANCE 


N 


Laboratory 


66.23 


50 


Audio 


68.26 


19 


Classroom 


69.92 


25 


Control 


59.04 


54t 



Between these groups, t tests were run on the mean performances 
to ascertain any significant differences. Independent t's were run 
on all comparisons except for that between the Laboratory and Con- 
trol groups as the subjects in these groups had been matched. A 
correlated t test was run on this comparison. The data are summa- 
rized in Table 5. 



TABLE 5? The t Values for Comparisons Between 





Mean~Criterion Test 


Performances 








t 


value 


df 






£ 


1. 


Laboratory and Control 


2.32 


49 


. 025 , 


one 


tailed 


2. 


Laboratory and Audio 


.51 


67 


n.a. 






3. 


Laboratory and Classroom .762 


73 


He 8 • 






4. 


Audio and Control 


2.19 


67 


.025, 


one 


tailed 


5. 


Classroom and Control 


2.74 


73 


.005, 


one 


tailed 


6. 


Audio and Classroom 


.15 


42 


n.s. 







A t test was also run on the criterion test scores of those 
subjects with I.Q.'s of 95 plus between the Laboratory and Control 
groups. The resulting t was 2.29 with 24 d.f., and was significant 
at the .05 level, one tailed. 






Criterion Test Components 

Figure 1 presents a frequency diagram of the criterion test 
scores of the Control and Laboratory groups. Although the ranges 
of scores for these two groups are almost identical, the Control 
group scores extend down twenty points below the Laboratory group 
scores. And, the scores from the Laboratory subjects extend upward 
twenty points above those of the Control group. 

To make a finer analysis of criterion test scores, the content 
of the criterion tests was analysed. This content analysis should 
lead to statements concerning which items on the test contributed 
to the overall significant difference. The criterion test was 
broken down into three major components. These components consist 
of items on the three topics taught in the unit . the Frequency 
Table, the Mode, and the Median. The Frequency Table items repre* 
sent 32.5 points, on the criterion test, the items on the Mode make 
up another 32.5 points, and the items on the Median account for 
34.5 points. 

In an attempt to account for the significant difference between 
the criterion test performances of the Control and the Laboratory 
experimental groups, test scores were broken down into measures of 
the three components. Table 6 presents scores based on the percent 
of errors on all items related to each conceptual component. These 
scores ere presented for both the Control and the Laboratory groups., 



TABLE 6* Percent of errors on the criterion test 

items measuring the concepts of the Median, 
the Mode, and the Frequency Table for the 
Control and the Laboratory groups. 



ITEMS 

Median 

Mode 

Frequency Table 



GROUP 

Laboratory 



46.96% 

25.22% 

19.20% 



Control 

56.00% 

27.65% 

25.94% 










Oft each category of teat items, the Control group has a higher 
percent of errors than the Laboratory group. For both the Control 
and Laboratory groups, the items on the Frequency Table yielded 

the least percent of errors while the items on the Median -led to 
the highest error scores. 

A further breakdown was made on these data by dividing the 
subjects in both groups on the basis of I.Q. This division was 
made at I.Q. 90. Table 7 presents these data. The scores are 
mean correct performances by group on the items related to the 
three conceptual categories of the criterion test. 



I 



TABLE 7 



I.r* 



Mean correct responses on criterion test 
items measuring the concepts of the Median, 
the Mode, and the Frequency Table by I.Q. 
for the Control and Laboratory groups. 



00.1 AQ 


L&lAii. nf 


Control 


P 

, 


Median 


16.17 


13.24 


Frequency Table 


30.71 


27.06 


oc 


Mode 


19.50 


18.47 


n.s. 


80-90 


Median 


11.66 


9.76 


n.s. 


Frequency Table 


26.97 


25.15 


n.s. 


Mode 


16.19 


15,50 


n.s. 



Between the Laboratory and Control groups t tests were run 

V N ** 

to test the mean difference on each criterion test component by 
I.Q. group. The only significant differences were for the higher 
I.Q. subjects (I.Q. 90 plus) on the items measuring the concepts 
of the Median and the Frequency Table. The rest of the comparisons 
yielded nonsignificant t's. 
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• Control Group 

• laboratory Group 




FIGURE Is Frequency diagram of criterion test scores 
for the Control and Laboratory Groups • 
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DISCUSSION 



The significant differences between mean criterion test scores 
for the Control and Experimental groups support the hypothesis that 
tutorial revision of instructional material will produce a more 
effective teaching tool than will workshop revision. There were 
no significant differences between the mean criterion test perfor- 
mances of the three experimental groups. The method of presentation 
of the tutorially revised material — printed format vs. audio 
format, laboratory setting vs. classroom — did not affect criter- 
ion test performance. The material taught as effectively in all 
three experimental group situations. 

Considering ;the level of the scores in the Control and E*£>eri~ 
mental groups, the workshop revision of the material used to instruct 
the Control group yielded scores no higher than 79.5. The tutorial 
revision, however, led to scores as high as 98.5. There was a 
similar difference on the lower end of the scale with the Control 
material yielding scores as low as 12.5, while the lowest score 
in the Laboratory group was 32.5. Not only was the workshop revised 
material a less effective instructional instrument when considering 
the mean scores, also it never led to scores which would ordinarily 
receive a good grade in a classroom. Only the tutorially revised 
material yielded scores that could be graded as either A, or B. 

From the data on the components of the criterion test, it 
can be seen that the r&dian was the most difficult topic for either 
group. The Mode was the second most difficult topic, and the Fre- 
quency Table was the easiest. Whether these error scores represent 
a measure of the basic conceptual difficulty of the three topics, 
or whether they merely reflect the ease with which either of the 
materials taught the topics is a moot point. 

When the Exp*iiraental and Control groups were broken into high 

I.Q. (90 plus) and low l.Q. (80-90) subjects, there was no clear' 

_ at tarn of relationship between IQ level and the relative effec- 
tiveness of tae tutorially- revised versus the workshop-revised 
materials. !/hile two significant differences between 'the two kinds 
of materials appeared in the high IQ group, all differences for 
both groups were in the same direction and of similar magnitude. 
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8 "° baSiS f ° r C ° nClUdin9 any corr ®l*tion between teaching 
effectiveness of materials and level of 1Q . This result encourages 

6 e ie ^ tutorial revision process produces study 

materials yielding test performances which are not correlated with 

IQ level and which may even tend to compensate for measured 
differences in IQ. 

While not reflected in the test data, the subjective exper- 
iences of the tutorial staff and the teachers from Simon Gratz 
were of considerable value in the growth of both young and old 
staff members. Asked to comment upon the value of their exper- 
iences during the pilot project, the teachers and their student 
consultants expressed enthusiasm for the tutorial procedure of 
analyzing and revising text materials. The study gave a 
critical awareness of the need to analyze student learning 
problems through face-to-face interaction and for subsequent 
tailoring of the learning experiences. The following are 
comments from the various staff members elucidating the advantages 
gained from participation in the study. 

"That which 1 formerly considered beneath mv 
H ? i now particularly stressed. 1 no longeTleava^mt 
information the student should have had. l make sure 

the^last*six°or ei^yeirs*" b * e " P#rt ° £ th ° curriculun > *or 

£° m to I^ ic 5 1 w ° rkod °» became entirely 

•MlgwS.'*" 3 *" 6 * i9ht - new “ t « r -**•" aabsrkinq ,m P u 

relationship between student and teacher?" - 1 
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The approach hypothesized as most productive of * «*»-.«,* 

i e S2»?nr Perie ? 0e tor deprived area 8tudenta f was ,t «pparently 

a learning experience for staff as well* PP y 
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RECOMMENDATIONS FOR SUBSEQUENT RESEARCH 

From the analysis of the concepts measured by the criterion 
test, it can be seen that the intergroup differences in student 
achievement ate due to the unequal teaching effectiveness of the 
workshop and the tutorially revised material covering the topics 
of the Median and the Frequency Table, The fact that- *h» material 
covering the node did not give rise to a significant difference 
(although £t gave rise to an apparent difference in the same 
direction) is difficult to interpret without further observation; 
one possible explanation is that the tutorially revised material 
on the Mode should have been carried through additional iterations 
to come up to the quality of the other tutorial material. Tutorial 
revision was stopped as soon as the students could respond cor- 
rectly to the diagnostic questions built into the text, whereas 
the ultimate comparison between materials was made on the basis 
of criterion test items, where a longer period of retention is 
a factor. If the tutorial sessions had been continued until a 
criterion of retention (rather than immediate comprehension) was 
reached, the tutorially- revised material would undoubtedly have 
been even more effective. 

Recommendation 1. In future, statistical comparisons should 
be made of test performances between one revision and the next 
on a group of students not previously used for the research. 

The results should indicate the statistical reliability of observed 
changes and will result in a better basis for determining when 
a sufficiently effective text has been produced. 

Recommendation 2. When tutoring the subjects whose behavior 
was used as a basis for revising the text material, it was 



discovered that they were extremely reluctant to verbalize any 
learning difficulties they encountered. It was necessary to have 
the students read the material aloud in order to make their reading 
difficulties obvious, and to build diagnostic questions into the 
text to constantly check on their comprehension of the material. 

The one-to-one tutorial session is extremely foreign to the students 
and calls for a level of responsiveness that they have never 
experienced before, it is likely that in ordinary classroom situ- 
ations, any spontaneous expression from the students, whether 
relevant to the learning situation or not, has been seriously 
discouraged, in subsequent studies, it is recommended that the 
tutorial sessions be conducted with two or three students at a 
time to reinforce and encourage each other. Another procedure 
to be tested, involves having the students individually tape 
record their reactions to the material, with some encouragement 
and privacy, the students may be able to critique the material 

and their insights can be analyzed and used in the next successive 
revision. 

Recommendation 3» For the final tutorial revision, an answer 
booklet was prepared that gives correct responses to every 
question included in the material. It was observed during the 
actual experimental run of the tutorial material, that the students 
would quite often merely copy the answers onto their copy of the 
text. Although the immediate feedback resultant from having 
the answer booklet is deemed important , a better procedure would 
be to break the unit into sections and hand out the answers at the 
end of each section. This procedure would forcejfchi students to 
respond, and still provide them with feedback. The feedback would 
be delayed but probably not long enough to cause any significant 
difference between the techniques. 



Recorendation 4. All tutorial sessions in all subject areas 
should be subjected to a standard procedural routine that includes 
observation, recording, classification, and indexing of the inter- 
actions among tutor, text, and student. But, in subsequent studies 
it is recommended that in each subject area, a classification system 
should be developed that is intrinsic to that area . Learning prob- 
lems experienced by the student must be classified in categories 
directly related to the text material. For example, the system 
growing out of the mathematics text material will, of course, be 
quite different from that growing out of the English material, 
and so on. 

Each category of learning problems arising from the textual 
material must have one or more referents in the literal record 
of the student’s interaction with the material. A category entitled 
for example, "confusion of median with mode should be keyed to 
actual occurrences of such confusion in the learning sessions. 

A complete system would be keyed to all such occurrences. 

Recommendation 5. A quite different classification system 
should be developed out of the behavior of the student that can 
be specified without necessary reference to the textual material. 
Both student expresses discouragement” and “ student gives wrong 
answer are categories that are common to all subject areas. 

This response classification system would tend to cut across the 
subject matter areas pretty generally and would add unique and 
indispensable information. 

Like the text-problem classification system, the response or 
behavioral classification system would be keyed to actual occur- 
rences in the literal record of the tutorial sessions. 

The keying and ? ross-referencing of both systems together 
would result in a comprehensive index that would make all the 
information immediately accessible and all the interactions among 
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events in the system subject to analysis. A crucial step in 
building the general index would be the development of a code 
for each category that would allow direct input from the entire 
system, or data bank, to a computer. Only by electronic data 
processing can full and practical use be mace of such an 
extensive bank of data as results from the procedures described. 

Recommendation 6. The data bank should be used, when 
reasonably complete for any one subject area, for two distinct 
purposes? 1) general research, and 2) instruction of teachers. 

e second purpose could he sceompiighoA by using the data bank 
as the basis for a simulated teaching situation, where typical 
problems of learners and successful handling of those problems 
by teachers would be displayed in a program responsive to the 
teaching behavior of the student teacher. Only an extensive and 
empirical data bank can support such a simulation approach. 



APPENDIX A; Sample Observations of Tutorial Session 



Subject — B. McL. 
Fifteenth Revision 
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3 November, 1963 




* 


NO. 


Test Reference 

* 


Area of Difficulty Student Behavior 


1 . 


paragraph A4 


finding a mode in a list 


“I don't understand" 


2. 


•1 


•• o ii 


Gives verbal defin- 
ition of mode 
rather than number. 


3. 


paragraph B1 

/ 

o 

/ 


finding a median in a list 


has difficulty 
deciding which 
number is in the 
middle of the list 


4. 


paragraph C6 


counting number of circles 
in a line 


hesitates over 
instructions. 


5. 


paragraph D6 


definition of median and 
mode 


reads material 
too fast without 
any comprehension 


6. 


paragraph FI 


•1/2 of 4 is 


can't answer 


1 o 


paragraph HI 


inventing a median in an 
even numbered list 


confused over 
procedure 


8. 


paragraph LI 


demonstration of crossing 
out and tallying procedure 
in making a frequency 
table 


confused over 
crossed out 
numbers in list 
of data. 


9. 


paragraph L 


filling in a frequency* 
table according to 
directions 


no need to read 
directions, can 
do it without 
them 


10. 


paragraph M 


II .1 


il . $3 


11. 


teacher question 
following 
paragraph 116 


what is the mode" 


starts to find 
the median. 


12. 


paragraph N3 


titling a frequency table 


can't fill in 
title of middle 
column 

(' tally marks' 1 ) 


13. 


paragraph 0 


titling a frequency table 


titles a "Table 
of Weights as 
a "Table of Ages" 


14. 


paragraph P 


making a frequency table 
from supplied data 


slow about titl- 
ing: forgets 
"Table 



J 3 



Test Reference 
paragraph Q 

paragraph K1 

paragraph R 

paragraph R6 

paragraph T 
paragraph T1 

paragraph U2 

paragraph U3 

paragraph V2 
paragraph X 
paragraph Y2 



Area of Difficulty 

frequency table of 
heights 



Student Behavior 

unfamiliar with 
notation for inches 
( " ) and feet ('), 



frequency table with 
instructions at side 
next to frequency column 



stumbles reading. 



“how many distances are 
there in the table', 
finding total to 
frequency column 

finding median in 
frequency table 



responds with the 
number of categories 
of distances. 

identifies middle 
tally mark as median 
rather than middle 
item. 



finding mode in table wrong answer. 

dividing total number of divides wrong 
frequencies by 2 

dividing total number no blanks for re- 

of frequencies by 2 and sponses in text so 
finding middle tally confused, 

mark 



finding median in frequency identifies 
table middle tally mark 

as median rather 
than middle item. 



ct i; 



91 



‘If both middle tallys 
are on the same line 
the median must be 
the number on that line 
to the (left, right) 



confused over 
format. 
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APPENDIX B* Original Text Presentation of the Topic 

“The Mode and Two Subsequent Examples from 

Revisions 13 and 23. 



I. Original Text Presentation 



Definition of Mode 

"The MODE is the most frequently occurring number in a 
group of numbers* For example, the mode of 93, 92. 93 
90, and 75 is S3* To find the mode easily, arrange the 
statistical data (numbers) in a frequency table* 



II. Text for 13th Revision 



Definition of Mode 

Here are some numbers:* 

3 , 4, 4, 4, 6 , 8 , 9, 9, 11 
The number 4 happens MOre times than any other score. 
It if called the MODE. 

Here are some words to help you remember*: MOre ■ MODE 

In the example above, the MODE is . * 
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23rd Revision 



Here are some stick men. 




If we wanted to describe this group of men, what could we say 
about them? Hot very much the way they look now, so let's 
line them up from shortest to tallest. 



t*e find the smallest one; John (67 inches tall) 

And the biggest one: Bill (79 inches tall) 

Now we can say something about the group. They are all either 
taller than John or shorter than Bill. 

What else could we say? If we put them all in order from 
smallest to tallest, we could say much more. 



ToKn 



(same height) 
tarry Rav 



roaaier 

*SoB 






Jamea~~ Bill 




Here are some very simple things we can now use to describe 
our group of stick men: 

1. There are seven of thorn. 

2. They are all between the heights of 67 to 79 inches. 

3. Larry and Ray are the same height. In Statistics 
we call their height (70 inches) the MODE. It is 
the most common or popular thing in a group. 






< 

APPENDIX B-l: Outline of Text Treatment as Taken from the 

Examples shown 



I. Original Text 

six exercises finding a mode in a frequency table 

two thought problems relating to the definition 
of the mode 

one question about the method of calculating the 
mode 

i.e., nine exercises all together 

II. 13th Revision 

two exercises finding mode in an ordered list 

two exercises finding the mode in an unordered list 

eleven exercises finding the mode in a frequency 
table 

i.e. , fifteen exercises all together 
three re-definitions of mode 



III. 23rd Revision 

five exercises finding mode in ordered list 
eight exercises finding mode in unordered list 
twelve exercises finding mode in frequency table 
i.e., twenty- five exercises involving mode 
six re-definitions of iso do 




APPENDIX C 

GENERAL EDUCATIONAL OBJECTIVES 
OF THE FIVE BASIC SUBJECT AREAS 



Brooks Foundation 
Proposal 902-2 



I. READING UNIT 



To be able to read and understand trade publications,, 
manuals, job sheets, want ads and newspapers. 

To be able to expand vocabularies, especially by using 
contextual clues to learn the definition of new words. 

To be able to locate personal and business listings in 
telephone directories. 

To be able to find places In the city on a map and to 
determine the best routes for reaching these places. 

To be able to extract essential information from catalogs, 
reference books, container labels and job specifications. 



II. MATHEMATICS UNIT 

Teach students addition, subtraction, multiplication and 
division of whole numbers with respect to the number line. 

Teach students the meaning of a fraction, and how to 
perform the fundamental operation of fractions. 

Teach students the meaninq of mixed numbers and their 
relations to fractions. 

To develop an understanding of arithmetical and mathematical 
principles and processes. 

To extend and strengthen the student's vocabulary of 
arithmetical and mathematical terms. 



III. SPEECH UNIT 

To teach the students to articulate words clearly. 

To teach students to speak In sentence form. 

To teach students to differentiate between casual and 
formal spoken engllsh. 

To teach students to hold sequential conversations. 

To teach students how to use a telephone effectively 
and how to behave at a job Interview. 



P**m*ee1 I02»l 



IV. WRITING UNIT 

To be able to revise paragraphs through the use of 
coded materials Including: 

Dictionary Skills 
Grammar 

To be able to Identify and write topic and clincher 
sentences. 

To be able to use connectives In writing. 

To be able to write descriptive* narrative* and 
expository paragraphs. 

To be able to complete job application forms* questionnaires 
and job reports In an acceptable manner. 



V. MANIPULATIVE SKILLS UNIT 

To develop the basic skills and knowledge required for the 
use of measuring Instruments. 

To learn how to make effective drawings and to read 
blueprints. 

To become familiar with techniques of assembling materials 
from printed and oral Instructions. 

To be able to Identify and analyze basic types of machlneso 

To beable to construct models from written specif leaf Ions 
and blueprints. 



START READING HERE . 
READ EVERYTHING IN QR0 
DO NOT SKIP ANYTHING. 
00 THE THINGS ASKED 
IE SURE YOU HAVE AN 



OF YOU 



BOOKLET 



In 

a . 

b. 

c. 



of you there should 



A measuring board 

A set of measuring blocks In a white box 

withVlettfi! flJii*’ 2 "k b,ue co1ored blocks 
on ‘each'on^of f ?hein* ed (B-l ) daSh a " d 3 number 



Look In your information booklet. 

Find the pictures of each of the above 
valuable information which should help 



and read the 
you do this 



GO ON TO THE NEXT PAGE 
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PAGE 9 



’ j"b W # V® questi0,is be,ow ,n oNer to get ready for 

Put your answers in the space below each question. 
First check your red, white and blue colored blocks , 
a. Mow many do you have? 



(11ke C B-l) 6d bl0ek has a letter and a number on 

What is the color of the side of the block 
which has this letter and number on It? 




c. Now draw that mark which looks like a letter. 
You can see It on every side of every block. 



ANSWER' — — — — 

N e_xt check your measuring blocks In the white box. 

d. What Is the color of the metal measurlnq 
blocks marked "1/16"? 



ANSWER — — 

G * of th ? se metal measuring blocks marked 

"1/16" do you have? 



ANSWER 



GO ON TO THE NEXT PAGE 

- 3 z,- 
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fc How many one««nch 

do you have? 



measurl ng 



blocks marked 



00 1 Gt 



afwpt” 



Mow many two~inch measuring 
do you have? 



blocks marked 



91 2 11 



ANSWER 



h. 



How many three~inch measuring 
do you have? 



blocks marked 



"3 



Answer 



3. 



c: 



Si;e Ce r.Ts°l5e d y °“ r me9SUr,n9 b04rd * 

The little arrow should be facing down (VI 

“ K:*, ;„!:**'• «« « >"• •~ir.su,, .... 

N ° W below. meaS “l !! 9 b04rd .? h0u ^ >ook like the picture 




4 ' box) S 1n?l tSe r s?o?! Ur1n9 b1 ° CltS (the b1ocks ,n the whit 




60 ON TO THE NEXT PAGE 
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sliding In 
until the lenght 



measuring blocks 1 
the measuring b 



§ Is the 



slot 
same a; 



6 , Add 



Use your fingertips to feel if they are even, 

up the whole numbers on the tops of the blocks, 
s is the number of whole Inches, 

7 * Then co “ nt “P Jhe number of the gold colored metal niece* 
and put this number on top of the number 16. ( number ) 

This is the number of sixteenths. 

8. Put both these numbers* together. 

sixteenths? 1e " 9th ° f the $1de#1n who,e "umbers and 

9. Put this number on the chart found on pag ejS/J. 

Put this in the right space on that chart. 



IF THF APotl } c ??I*£ L ! AR F0LL0W THE EXAMPLE GIVEN BELOW 
H to A step J S io CLEAR SKIP THE example AND 60 DIRECTLY 



EXAMPLE 



Ut * 9 your a»*aurlng board looked nke thio: 




1* Add up the whole numbers ( 3 and 2 le 

2. Count the nuaber of gold colored octal placas ( whleh h... 4 ■a. 
Put this number over 16 ( which would be 3/16) S ^ # 



GO ON TO MBIT PACK 



- 3 + 
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PAGE 12 



sho M lS e h, e ve tW 7 S7llf r (7 and s/16) you 



11 . 

12 . 

13. 

14. 



HHl-ll the end of the example . 

« 

Do the smae thing to the white and blue sides of block 
“2", t “3" l *and ”1/*16" ° n,y the me#sur1 f9 blocks marked “1 



Put these answers on the chart found on page #13. 

of*blSck!! e Bf2" e B t a 1n |! 2° J"l r S d i wh,te ■"<* b loe sides 
or oiocks B-Z, B-3, B-4, B-5, B-6, B-7; and B-8. 

Put these answers on the chart found on page #13. 

empty s P #ces on your chart have been filled 
in as best you can, do these things: " e<l 



a. ) 

b. ) 

c. ) 



whitepox 11 measur1n 9 ^ocks back Into the 

Be sure that all the measuring blocks are there 
Tell your teacher If any are Slsslng. 

cSlireS blocks?'* *" th * wMte an<1 b,ue 



Y00 HAVE NOW COMPLETED JOB #2 . 

FINISHED . 



TELL YOUR TEACHER YOU APE 



0 



- 56 - 



m 





PA6E 13 . 





» 

I 











£ 



V*- 



AFFE19DI3C Ds Ijidcninf Difficult!.©© Associated with th© 

Operation of Making a Frequency T ab le and 
with the Concept of the laode 

Operations Making a Frequency Table 
Learning Difficulties; 

1 e Titling the table r 

a. forgets column titles 
bo forgets title to whole table 
c. incorrectly titles columns and whole fable 
3® Filling in the item column. 

a. doesn't table items of category in order of 
increasing magnitude 

b. missed one item category 

c. left out item column entirely 

d. puts all items in column rather than item 
categories 

3® Tally mark column - 

a. doesn't tally each item 

b. tallies items randomly rather than counting them 

c. not crossing out items as tallying 

d. left out tally mark column entirely 

e. tallies one category twice 
f® puts tallies in. wrong column 

g. puts numbers in columns rather than tally marks 

h. makes tallies correspond to items rather than 
to frequencies 

i. crossing out item categories in table rather 
than list as tallying 

j. crossing out items as filling in item column 
rather than in tally mark column 

4 • Frequency column 

a® doesn't table frequencies instead numbers 
each line 

b. left out frequency column 

c. put frequencies in wrong column 

d® added total of frequencies incorrectly 



5 . Data: 

a. doesn't put data in discrete categories for tabling 

b. sets up titles but doesn't fill in data 

6. Format; 

a. sets table up vertically rather than horizontally 

Concept r The Mode 
Learning Difficulties . 

1 • Confuse !>ode with the total of the frequencies* 

2* Confuse Mode with the frequency that occurs most often* 
3* Confuse fidde with the highest frequency* 

4* Confuse Mode with the highest item category « 

5. Confuse iiode with the median. 



